In this work, we introduce temporal hierarchies to the sequence to sequence(seq2seq) model to tackle the problem of abstractive summarization ofscientific articles. The proposed Multiple Timescale model of the GatedRecurrent Unit (MTGRU) is implemented in the encoder-decoder setting to betterdeal with the presence of multiple compositionalities in larger texts. Theproposed model is compared to the conventional RNN encoder-decoder, and theresults demonstrate that our model trains faster and shows significantperformance gains. The results also show that the temporal hierarchies helpimprove the ability of seq2seq models to capture compositionalities betterwithout the presence of highly complex architectural hierarchies.
展开▼
